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Executive Summary 


In late 2014, the Massachusetts Department of Early Education and Care (EEC) was awarded a federal 
Preschool Expansion Grant (PEG) to support the expansion of high-quality early childhood education in 
Massachusetts to four-year-old children from low-income families. Although the Massachusetts PEG 
model supports and encourages local program development and adaption, each PEG program is expected 
to focus on the following quality elements (Exhibit E.1): 


Exhibit E.1: PEG Model Quality Elements 


A collaborative local governance structure designed to oversee implementation and work on systems coordination for 
all children in the community; 
( ) 


| 2. | Full-day, full-year programming (at least 8 hours/day, 12 months/year); 
3. 
; 


Professional development training and coaching for teaching staff, and other supports for planning and implementation 
of curriculum, in collaboration with the LEA; 

Family engagement activities, including support for kindergarten transition and resources about child development; 
Comprehensive services including services addressing health, mental health, and behavioral needs for all families; 


Inclusion of students receiving special education support; and 
Efforts to build linkages with services for children from birth to age 3 as well as connections with elementary schools. 


The Year 2 PEG implementation study focuses on the second year of supports and outcomes for teachers, 
classroom quality, and supports for families. Overall, compared to its first year of implementation, the PEG 
programs in Year 2 (which was the 2016-17 school year) appear to have made progress towards delivering 
the key quality components of the PEG model at a consistently high level across all 48 classrooms. 


Ea 
4. 
5. | 
6. | 
a 
| 10. 
11 
12, 


Findings on Year 2 Educator Supports 


In the second year of PEG implementation, programs continued to maintain highly-educated teaching staff. 
PEG programs retained most of their teachers from Year 1 to Year 2; 77 percent of lead teachers and 70 
percent of non-lead teachers continued teaching in the second year of the program. 


Programs continued to provide professional development opportunities to educators through training, 
coaching by local school district staff, and formal release time for teacher planning.' 


e Training. Compared to Year 1, in Year 2, PEG lead and non-lead teachers received, on average, more 
training. Also, the differential in the amount of training between lead and non-lead teachers that was 
identified in Year | narrowed in Year 2. 


e Coaching. Nearly all lead and non-lead teachers reported receiving coaching in Year 2, and the 
proportion of teachers that received coaching was higher in Year 2 than in Year 1. In both years, most 
coaching was provided by PEG coaches employed by the local school district. 


'Tn Year 1, only four of five communities provided PEG coaching. 
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Release time for planning. In Year 2, more lead teachers reported having formal release time for 
instructional planning, although the majority still reported that the amount of release time was not 
sufficient to complete their job responsibilities. 


In Year 2, PEG programs responded to implementation challenges identified in Year 1, and most notably 
expanded and improved some of the professional development opportunities to educators. In particular: 


PEG communities and programs intentionally increased the alignment across the different forms of 
professional learning (i.e., training and coaching) and the coherence of the professional learning, 
classroom curriculum, and assessments. 


PEG districts and programs expanded the professional learning opportunities for non-lead teachers. 


PEG programs focused more on establishing a professional learning community for teachers, which 
represents a potentially powerful avenue for sustaining improvements in programming and instruction. 


Teacher assessment of the usefulness of the training and coaching remained decidedly mixed. PEG 


districts and programs may want to explore further which professional opportunities would be more 
meaningful for improving their practice. 


Findings on Year 2 Classroom Quality 


Observations of the 48 PEG classrooms in Year 2 using the CLASS, a standardized measure of overall 
instructional quality with a focus on interactions among teachers and children in the classroom, showed 
overall improvements compared to Year | (Exhibit E.2). 


For the Classroom Organization domain, the average CLASS score across all PEG classrooms rose 
from 5.2 to 5.7, which represents a significant increase. Substantially more classrooms were rated as 
high quality on this domain in Year 2 (75 percent) than in Year 1 (40 percent). 

For the Emotional Support domain, the Year 2 score (5.9) was more similar to the Year 1 score (5.7), 


and represents a significant increase. The proportion of classrooms rated as high quality increased from 
60 percent to 80 percent. 


For the Instructional Support domain, although the proportion of classes rated as moderate quality 


increased slightly from 75 percent to 79 percent, the average score remained relatively unchanged from 
Year 1 (3.2) to Year 2 (3.1). 
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Exhibit E.2: CLASS Average Total and Domain Scores by PEG Community and PEG Statewide, 
Winter 2017 (n = 48 classrooms) 


Total Score 


7 
6 
49 5.1 

5 47 ae 

4 

2 

1 |__| 

Boston 


5.2 
49 
a Holyoke Lawrence Lowell Springfield State _e 


Emotional Support Domain Classroom Organization Domain Instructional Support Domain 
7 7 7 
6 (88 sao oa 58 59 6 se s7 80 59 57 7 
53 
5 5 5 
4 4 4 34 34 35 
34 
3 3 3 3.0 29 
; : : fi a 
1 1 1 
Boston Holyoke Lawrence Lowell Springfield State Boston Holyoke Lawrence Lowell Springfield State Boston Holyoke Lawrence Lowell Springfield State 


4 Boston=15, Holyoke=4, Lawrence=10, Lowell=8, Springfield=11 


Improved scores from Year 1 to Year 2 at least in part reflected development over time in teachers who 
were there for both years. The majority of lead teachers were retained in PEG classrooms from Year 1 to 
Year 2. Excluding new teachers from the analysis, the average scores improved on all three domains 
(Exhibit E.3). 


Exhibit E.3: Percentage of PEG Classrooms in Different Quality Categories by CLASS Domain, 
Winter 2017 and Winter 2016, Classrooms with Same Teacher in 2016 and 2017 (n = 
36 classrooms’) 


Classroom Organization - 2017 


Avg=5.8 out of 7 
Avg=5.3 out of 7 


Classroom Organization - 2016 


Emotional Support - Avg=6.0 out of 7 


Avg=5.7 out of 7 


Emotional Support - 


Instructional Support - 2017 
Avg=3.2 out of 7 


Instructional Support - 2016 Avg=3.2 out of 7 


Low (1-2) Moderate (3-5) High (6-7) 


aBoston=9, Holyoke=4, Lawrence=8, Lowell=6, Springfield=9 


Abt Associates Year 2 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report J pg. iii 


EXECUTIVE SUMMARY 


The findings on classroom quality suggest that there is still some work to do to identify which professional 
learning opportunities will significantly strengthen instructional quality (e.g., support for teachers to 
provide more individualized instruction). This challenge is not unique to PEG, as the instructional support 
domain tends to be lower and the hardest to improve in the broader early childhood literature. 


Findings on Year 2 Family Supports and Comprehensive Services 


Community and program leadership made efforts in Year 2 to increase the level of supports for families 
and to introduce more intentionality in the focus of the supports. Overall, family engagement was perceived 
as an area of improvement by a majority (65 percent) of program directors in Year 2. 


PEG programs reported focusing less on individual parent social activities and events and focusing more 
on an approach to building parent capacities in Year 2, not only to increase home support for child learning 
and development, but also to enhance parent mental health as well as parenting and broader skills for the 
current job market. A high proportion of teachers (74 percent) reported participation in home visits, which 
raises the potential that deeper connections are being made between the classroom and the home. 


The parent-program connection is an area that merits additional exploration for the PEG program. Although 
challenges remain with engaging and serving all families, given the other demands that many PEG parents 
have, universal participation may be unrealistic, yet programs should continue to be creative about ways to 
deliver programs and supports to parents that are flexible and responsive to different parent schedules. 


Conclusions 


A continuing theme of PEG implementation is the notable variation both across and within communities in 
the reach and intensity of supports for educators and for families. More needs to be learned about the extent 
to which variation in supports and services is based on (a) differences across programs in the intentional 
emphasis on different parts of the PEG model; (b) differences within programs in the level of need of 
individual educators and parents that may arise because of scheduling challenges, educator or parent 
motivation, or the like; and/or (c) structural and capacity challenges in fully implementing some or all of 
the PEG supports. 


Overall, it appears that the key quality components of the PEG model are being implemented with 


increasing intensity. Whether or not the PEG model impacts child outcomes will be better understood in 
spring 2018, when impact study findings from the PEG evaluation will be available. 
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Introduction 


In late 2014, the Massachusetts Department of Early Education and Care (EEC) was awarded a federal 
Preschool Expansion Grant to support the expansion of high-quality early childhood education in 
Massachusetts to four-year-old children from low-income families. Shortly thereafter, EEC contracted an 
independent research firm, Abt Associates, in partnership with the University of Massachusetts Boston, to 
conduct a rigorous external evaluation of the program’s implementation, outcomes, and cost-effectiveness 
over its first four years. That evaluation, now in its third year, includes four components: 


| 2015-16 | 2016-17 | 2017-18 | 2018-19 
Implementation study of the PEG quality components 


| Longitudinal study of outcomes for PEG children and families | | SY | YT 
[impact siivotetecsionPEGchiomen fT EC yf ¥| 
| Cost studylcostetfectiveness study | CP 


After introducing the PEG program (Chapter 1), this report describes the second year of supports and 
outcomes for teachers (Chapters 2 and 3), classroom quality (Chapter 4), supports for families (Chapter 5), 
and concludes with a discussion of overall implementation progress in Year 2 and next steps for Year 3 
(Chapter 6). 


In addition to describing the /evel of implementation of key PEG quality components, Chapters 2 and 5 also 
note changes in implementation progress or strategy from Year | to Year 2, and the extent to which 
programs were able to address challenges identified in Year 1 during Year 2. 


Taken together, this report provides EEC and the larger PEG community with an important opportunity to 
learn from implementation successes and challenges experienced by the PEG program over the past year 
and can inform policy and programmatic decisions about further expansion of high-quality state-funded 
preschool. 
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1. Overview of the PEG Program and Evaluation 


1.1. PEG Program Model & Rationale 


For the past two years, Massachusetts has used federal PEG resources granted to the Commonwealth in late 
2014 to fund expansion of full-day, full-year preschool for four-year-old children through public-private 
partnerships between five local education agencies (referred to as public school districts) and licensed early 
learning providers (ELPs). The five communities that are participating in PEG are Boston, Holyoke, 
Lawrence, Lowell, and Springfield. In an effort to expand access to preschool for the most vulnerable 
families, children are required to meet the following criteria to be eligible for the program: 


e The child must have reached his/her fourth birthday by the beginning of their preschool year and be 
eligible for kindergarten in the following September; 


e Their family must reside within the boundaries of the public school district; 


e In four of the five communities, the child has not previously been enrolled in a licensed early learning 
setting; and 


e The family income must be below 200 percent of the federal poverty line. 


Four of the five PEG communities (all except Boston) target and primarily serve children who have never 
before been enrolled in licensed early education of any kind (including both center-based programs and 
family child care homes). In Boston, many children in PEG classrooms were already attending licensed 
care settings prior to their PEG preschool experience. 


Local PEG fund allocations are determined by using the Chapter 70 foundation per child allocation for 
preschool as a baseline and adjusting upwards to account for the PEG program’s extended hours per day 
and increased services. The funding mechanism is also designed to ensure a minimum investment in the 
smallest community (Holyoke) and a corresponding ceiling—adjusted for the high cost of living—for the 
largest community (Boston). 


The five participating school districts are granted PEG funds directly and then subcontract with community- 
based ELPs to provide direct services to children and families. As Exhibit 2.1 shows, each district 
subcontracts with two or more ELPs, including at least one Head Start agency and at least one community- 
based agency. Some ELPs operate PEG classrooms in a single location (referred to as a ‘center’ in this 
report), whereas others operate PEG classrooms across multiple locations. Prior to the PEG grant, all 
participating ELPs had experience administering preschool classrooms and managing the licensing of 
facility space. 


Exhibit 2.1: Number of PEG Participating Organizations and Classrooms by Community, 2016-17 


Public School District # of ELPs # of PEG Centers | # of PEG Classrooms 


fowl 


4One ELP operated PEG classrooms in two communities (Springfield and Holyoke). 
>In Springfield, three ELPs jointly operated one of the four centers. 
¢|n Lowell, two ELPs jointly operated one center. 
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1.2. Structure of Program 


Although the Massachusetts PEG model supports and encourages local program development and adaption, 
each PEG program is expected to focus on the following quality elements (Exhibit 2.2): 


Exhibit 2.2: PEG Model Quality Elements 


A collaborative local governance structure designed to oversee implementation and work on systems coordination for 
all children in the community 


3. 
; 
Professional development training and coaching for teaching staff, and other supports for planning and implementation 

of curriculum, in collaboration with the LEA; 
Family engagement activities, including support for kindergarten transition and resources about child development; 
Comprehensive services including services addressing health, mental health, and behavioral needs for all families; 


Inclusion of students receiving special education support; and 
Efforts to build linkages with services for children from birth to age 3 as well as connections with elementary schools. 


By the end of the grant period, PEG centers are also expected to attain the highest rating (Level 4) in the 
QRIS or Level 3 and National Association for the Education of Young Children (NAEYC) accreditation. 
Within the PEG model framework, districts and ELPs are provided with some degree of flexibility 
regarding the specific approaches they take to implement each element mentioned above. Some (though 
not all) of the ways in which PEG communities and/or programs differ are listed below: 


~ 


e Location of services (as already noted, some ELPs co-locate all PEG classrooms within one center, 
while others provide services across various centers in the community); 


e Approach to collaboration with other community providers; 

e Extent to which additional compensation of PEG teachers is greater than the typical salary in local 
community-based programs; 

e Curriculum/a used in the classrooms; 

e Focus and design of professional development and coaching supports offered; and 


e Focus of family engagement efforts and the provision of comprehensive services. 


The requirements guiding the PEG program model are intended to ensure the delivery of high quality 
ingredients and supports that research has repeatedly shown are likely to improve child outcomes, especially 
for children at risk for academic failure. The PEG model (Exhibit 2.3) is intended to achieve a certain level 
of quality in instructional and emotional supportiveness, classroom organization, and learning resources. 
Through these program characteristics, PEG programs are expected to achieve positive outcomes for 
educators, for parents and children, and for the larger system that supports early education in PEG 
communities. 
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Exhibit 2.3: Theory of Change for Massachusetts Preschool Expansion Grant 


State-Level Inputs/Activities 


Provide support, coordination , and 
technical assistance to LEAs and 
programs 


Collaborate and coordinate with other state 
agencies around early education policy 


Abt Associates 


School District (LEA) 
Inpurs/Activities 


Manage and oversee PEG subgrant 
Coordinate collaboration among 
directors of ELPs and other local child- 
and-family-servicing agencies 

Lead coaching for PEG educators 
Responsible for obtaining state IDs for 
children 

Lead and/or help with recruitment of 
PEG families and children 
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Program (ELP)-Level Inputs: 
Implementation 


* Curriculum and instruction aligned to 
state standards 

* Formative assessments to guide 
instruction, communication with families 


Professional development and coaching 
for PEG teachers 


Family engagement activities including: 
Parent education and home supports 
for learning 
Parent involvementin program and 
classroom activities 
Communication with families about 
child progress 


+ Link families/children with 
comprehensive wrap-around services 
for development and family needs 

* Provide or coordinate services for 
special needs children 

+ Kindergarten transition supports for 
PEG families 


Alignment of programming with state QRIS 
standards 


Participation in collaboration and 
coordination activities with other early 


education providers in community, 
including LEA 


Program Outcomes 


Intermediate 


Teacher Outcomes 

* Stronger instructional skills 

* Increased engagement with 
professional development 
Reduced turnover 
Higher sense Of efficacy and job 
commitment/satisfaction 
Increased knowledge of high-quality 
instruction, especially for high-needs 
subgroups 


Classroom Outcomes 

* High-quality instruction and 
programming 

+ Effective instructional strategies for 
diverse learners 


System Outcomes 

+ Increased access to high-quality early 
education for underserved 
populations in the community 

* Stronger, more connected early 
childhood education system 


Child and Family 


Child Outcomes 
* End of preschool and early elementary 
school child outcomes 
* Early academic skills 
* Socio-emotional skills 
* Promotion 
Reduced suspension 
Attendance 
IEP Status 


Family Outcomes 

+ Satisfaction with program engagement 
activities 

+ Increased ability to support child 
success in kindergarten and beyond 

* Increased ability to support child’s 
learning in home 

* Increased access to comprehensive 
services 
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1.3. Program Demographics 


The PEG program has been successful at enrolling children who meet the program eligibility criteria 
(Exhibit 2.4). In the second year of PEG: 


e PEG families reported incomes below 200 percent of the federal poverty level, which is a requirement 
to enroll ina PEG program. Moreover, 66 percent of PEG families reported incomes below 100 percent 
of the 2017 federal poverty level for a family of four ($24,600). The average PEG family income was 
$19,203 dollars per year. 


e Over 90 percent of PEG children were from racial and/or ethnic minority groups. Overall, over half of 
children served in PEG classrooms in 2016-17 were Hispanic (52 percent); the proportion of Hispanic 
children ranged from 27 percent of PEG children in Boston to 99 percent of PEG children in Lawrence. 
About one-quarter of children (28 percent) served were Black, with the proportion of such children 
ranging from zero percent in Holyoke to more than 61 percent in Boston. 


e Slightly less than half (44 percent) of PEG children lived in households where English was not the 
primary language spoken. In two communities (Lowell and Lawrence), fewer than one-third of children 
reported English as their home language. In Lawrence, the majority of children were from Spanish- 
speaking homes, and in Lowell, children reported speaking a wide range of languages. 

e PEG classrooms served a small population of children with Individualized Education Program (IEP) 
plans, which are developed for each public school child eligible for special education to guide his or 
her services. The goal is to target enrollment so that at least seven percent of the children in each PEG 
classroom have an IEP; in 2016-17, close to six percent of children had one. 


Exhibit 2.4: Demographic Characteristics of PEG Children Overall and by Community, 2016-17 


Number and Percentage of Children 
OverallPEG | Boston | Holyoke | Lawrence | Lowell | Springfield | 


Race/Ethnicity 


| Caucasian—non-Hispanic | 51 | 6% | 5 | 2% | 0 | 0% | 0 | 0% | 29 | 18% | 15 | 9% | 
pBlack 29 | 26% | tot | 61% | 0 | o% | 4 | 1% | 20 | 12% | 37_| 22% | 


[Asian-American | 68_| 9% | 9 | 3% | 0 | 0% | 0 | 0% | 59 | 36%] 6 | 4% | 
|Twoormoreraces | 20 | 3% | 7 | 3% | 0 | 0% | 0 | 0% | 7 | 4% | 6 | 4% | 
omer | th | 2% | 9 | om | 5 | 8% | 0 | O% | 0 | O% | 0 | | 
Primary Home Language | | 

a 
Othera 87 | 11% 15% 0% 1 1% | 43 | 27% 3% 


Notes. Data obtained from EEC data for all 48 PEG Soone during a 2016. Percentages may not add up to 100 because — are 
rounded to the nearest whole. 

4 Other common languages included Cape Verdean, Chinese, and Haitian Creole (Primarily in Boston), and Portuguese, Vietnamese, and 
Arabic (primarily in Lowell). 
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1.4 Overview of Year 2 Evaluation 


1.4.1. Data Sources and Sample Description 


The data presented in the remainder of this report are drawn from multiple sources including: 


e An online survey of all PEG teachers and assistant (referred to as ‘non-lead’) teachers; 


e Interviews with PEG program directors and other program-level instructional leadership (referred to as 
education directors), PEG coaches, and PEG family engagement specialists/support staff; and 


e Direct observations of all 48 classrooms to assess instructional quality. 
Response rates were high across the different data sources. 


The teacher survey was sent to 122 PEG lead and non-lead teachers in spring 2017. Eighty-four teachers 
completed the survey, for a response rate of 69 percent (Exhibit 2.5). The response rate was higher for lead 
teachers than for non-lead teachers. Nearly all lead teachers responded? (90 percent), representing 44 of 48 
PEG classrooms, and 38 of 71 non-lead teachers responded (54 percent), representing 25 of 48 PEG 
classrooms (Exhibit 2.6). Exhibit 2.5 summarizes survey responses by community. Three classrooms—one 
each in Boston, Lowell, and Holyoke—were not represented by either a lead or non-lead teacher. 


Exhibit 2.5: Overall Teacher Survey Response Rate, Spring 2017 


| ead | Nometead =| AC 


% of PEG 
% of PEG Lead Non-Lead % of All PEG 
Respondents Teachers Respondents Teachers Respo Teachers 


ndents 
Boston | _—t6e_—s | —sa%@ =| 
[Lawrence | = 9 | 90% =~ | 4 


4 Three classrooms in Boston had more than one teacher who was considered to be a ‘lead’; in two of those classrooms, both lead teachers 
completed the survey. 


2 Though there are 48 PEG classrooms, three classrooms had two lead teachers in each, for a total sample of 51 lead 
teachers. 
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Exhibit 2.6: Proportion of Classrooms with At Least One Teacher Survey Response, Spring 2017 
At Least One Lead Teacher At Least One Non-Lead Teacher 
Respondent Respondent 
Total PEG % of 
Classrooms # Classrooms Classrooms # Classrooms 


% of 
Classrooms 


40% 
100% 
30% 
63% 


75% 
ea 
25 ou 


Notes. The following four ELPs had a classroom without a lead teacher respondent: ABCD, GLCAC, Little Sprouts, and VOC. The following 12 
ELPs had a classroom without a non-lead respondent: ABCD (3), Boys Girls Club of Dorchester, Catholic Charities, Community Group (3), CTI 
(2), Ellis Memorial, GLCAC (4), HCS-HS Springfield (2), Little Sprouts, Paige, YMCA of Greater Boston (2), and YMCA of Greater Springfield 
(2). ABCD, GLCAC, and Little Sprouts each had one classroom with no respondent. 


The participation rate for interviews was high for each respondent type, though somewhat lower for family 
support staff than for the other staff groups (Exhibit 2.7). Observations were conducted during the late 
winter of 2017 (February-March) and interviews were conducted during the late spring of 2017. 


Exhibit 2.7: In-Person Interviews Conducted, Spring 2017 


| === | Coaches ~—s | _—_—Program Directors _| Family Support Staff Education Directors 
% of All 
PEG 
Family % of All 
Support PEG Ed 
| Boston | 


% of All 
PEG 
Coaches 


% of All 
PEG 
Directors 


100% 
100% 
Springfield 100% 
@ The Year 2 coaches who worked with programs for the first part of the year (as well as during Year 1) left the district mid-year. Although new 
coaches were hired in March-April 2017, they were not interviewed. 
> There was overlap in family support staff between Springfield and Holyoke. 
© These three staff were hired in late Spring 2017 and include one family support staff per ELP along with a supervisor. 


4A zero in this column does not indicate that there were not education directors in these communities; only a sample of the education directors 
were interviewed to help capture the professional development activities in each community. 


Boston | 4 | 100% | 9 | 75% | 3 | 
aa 0 
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2. Findings on PEG Professional Learning Opportunities 


The PEG program model seeks to provide high-quality instruction and learning environments through 
effective systems, structures and practices. To this end, PEG programs employ well-educated staff who are 
provided with levels of compensation that maintain parity with the local school districts. Also, districts and 
programs have worked to build the instructional capacity of PEG educators through multiple job-embedded 
professional learning opportunities, including training and coaching, and paid release time for instructional 
planning and collaboration. 


In the second year of PEG implementation, programs continued to maintain highly-educated teaching staff 
and provided training, coaching, and some formal release time for teacher planning.* Responding to 
challenges identified in Year 1, PEG districts and programs increased the alignment across the different 
forms of professional learning (i.e., training and coaching) and the coherence of the professional learning, 
classroom curriculum, and assessments. PEG districts and programs also expanded the professional 
learning opportunities for non-lead teachers, although in most cases, non-lead teachers still received fewer 
supports than lead teachers. In Year 2, PEG districts and programs focused more on supporting teacher 
collaboration around improved instruction and program quality and establishing a professional learning 
community for teachers. Districts and programs continued to face challenges with providing professional 
supports focused on the instruction of special needs children. 


This chapter provides details about the professional learning opportunities for PEG teachers in Year 2 
including training, coaching, paid release time, and teacher collaboration. Changes observed between Years 
1 and 2 are highlighted, with a focus on Year | implementation challenges that were successfully addressed 
and challenges that persisted or were newly identified in Year 2. 


2.1. Level of Education of PEG Teaching Staff in Year 2 


PEG programs retained most of their teachers from Year 1 to Year 2; 77 percent of lead teachers and 70 
percent of non-lead teachers continued teaching in the second year of PEG. Because of the low rate of 
turnover, the characteristics of the PEG teaching staff in Year 2 remained generally similar to Year 1. In 
Year 2, almost all lead teachers had at least a bachelor’s degree and almost all had multiple years of 
experience as early childhood teachers. Of note, compared to Year 1, in Year 2, more PEG non-lead teachers 
had a bachelor’s degree, and more non-lead teachers were concurrently enrolled in higher education 
(Exhibit 3.1). 


3 Tn Year 1, four of five communities provided PEG coaching. 
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Exhibit 3.1: Characteristics of PEG Teachers, 2015-16 and 2016-17 


High school degree 0 0% 0 0% 10 31% 12 32% 
Two-year degree/vocational degree 0 0% 2 4% 16 50% 15 39% 
Bachelors 30 77% 33 72% 5 16% 10 26% 
Masters 9 23% 11 24% 1 3% 1 3% 
White Non-Hispanic 18 44% 10 30% 
Black Non-Hispanic 9 22% 9 27% 
Not collected Not collected 
Hispanic ) 22% 10 30% 
Other 5 12% 4 13% 
1-2 years 3 8% 10 24% 7 21% 12 38% 
3-5 years 12 31% 14 34% 14 42% 11 34% 
6-10 years 12 31% 8 20% 5 15% 3 9% 
More than 10 years 12 31% i) 22% i 21% 6 19% 
Fluency in Spanish 
Spanish 
Taking classes/Working other job 
Currently taking classes 8 21% 10 22% 14 42% 23 61% 
Currently employed outside of PEG 8 21% 4 9% 5 15% 6 16% 


2.2 Training Opportunities for PEG Staff in Year 2 


This section combines teacher-reported information on training received (via teacher surveys) and 
perspectives on training from program directors and education managers (via in-person interviews). 


2.2.1. Amount of Training in Year 2 


In Year 2, half of lead teachers and 42 percent of non-lead teachers reported receiving at least 21 hours of 
professional training (Exhibit 3.2). Compared to Year 1, it appears that the number of hours of training 
received by PEG teachers increased, on average, for both lead and non-lead teachers. Also, the differential 
in the amount of training between lead and non-lead teachers narrowed. Whereas in Year 1, twice as many 
lead teachers as non-lead teachers received at least 21 hours of training, the percentages in Year 2 are 
substantially more similar. These changes indicate that an issue raised in Year | — the need to increase the 
increase the professional learning opportunities for non-lead staff - was beginning to be addressed. 


At the same time, there were still some educators who reported receiving only a few hours of training over 
the course of Year 2. The PEG program does not establish specific thresholds for the expected amount of 
training staff should receive, and there is not an accepted optimal threshold in the field. Therefore, it is not 
clear whether the program is providing a level of training sufficient to lead to continued improvements in 
teachers’ instructional practices. It seems unlikely that ten or fewer hours of training over the course of a 


Abt Associates Year 2 Massachusetts Preschool Expansion Grant (PG) Evaluation Report | pg. 9 


PROFESSIONAL LEARNING OPPORTUNITIES 


year, as was reported by nearly 30 percent of lead teachers and 24 percent of non-lead teachers in Year 2, 
is sufficient to achieve meaningful change in practice. 


Exhibit 3.2: Hours of Training Received (Not Including Individual Coaching), 2015-16 and 2016-17 


0% 0 0% 


None 0 0% 1 2% 0 
1-5 hours 2 8% 3 7% 3 25% 5 13% 
6-10 hours 5 20% 9 20% 3 25% 4 11% 
11-20 hours 8 32% 10 22% 3 25% 13 34% 
21-40 hours 6 24% 15 33% 3 25% 15 39% 
More than 40 hours 4 16% 8 17% 0 0% 1 3% 
Average # hours 53 hours sabe taiROLIS 21-40 
ours? hours 


a A mean value could not be computed in Year 2 since teachers selected a range of hours versus indicating a particular value; the range 
listed here therefore represents the most common response selection. 


In Year 2, similarly to Year 1, the amount of training reported by teachers varied by community (see Exhibit 
3.3). In Year 2, in three communities (Holyoke, Lowell, and Springfield), the majority of lead teachers 
reported receiving at least 21 hours of training, while in the other two communities (Boston and Lawrence), 
less than half of the lead teachers reported this many training hours. The pattern was similar for non-lead 
teachers in Year 2, except that a higher number of training hours was reported by non-lead teachers in only 
two of the communities (Lowell and Springfield) (Appendix Exhibit A. 1) 


Exhibit 3.3: Hours of Training Received by Lead Teachers (Not Including Individual Coaching) by 
Community, 2016-17 


100% 
80% 
60% 
40% 


* aiitlln f . if 
0% _allll anni i ai wt mob 


Percent of Lead Teachers 


Overall PEG Boston (n=16) Holyoke (n=3) — Lawrence (n=9) Lowell (n=7) Springfield (n=11) 


(n=46) Community 


MNone m1-S5hours 6-10 hours m#11-20hours #21-40hours m>40 hours 
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2.2.2 Content and Perceived Effectiveness of Training in Year 2 


In Year 2, nearly all lead teachers reported training related to content instruction, guiding and interacting 
with young children, strategies for supporting children’s language-rich interactions and children’s socio- 
emotional development, and working with families (Exhibit 3.4). Fewer lead teachers reported training on 
working with children with special education needs, although the overall percentage of lead teachers (65 
percent) represents an increase over Year 1, which may indicate a more concerted effort in Year 2 to provide 
needed training in this topic. 


Generally, less than half of lead teachers rated any of their training as “very effective” (Exhibit 3.5). The 
proportion of teachers that judged the training to be effective did not differ much across topic areas, 
although the fewest number of teachers rated the training related to children with special education needs 
as effective. Non-lead teachers were more positive about the training; on most topics, more than half of 
these teachers rated their training as very effective. 
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Exhibit 3.4: Topics of Training Received by Lead and Non-Lead Teachers, 2016-17 


% Lead Teachers Who % Non-Lead Teachers 
Reported Receiving Any Reported Receiving Any 
Training on Topic Training on Topic 

Training Topic (n = 42-46 teachers) (n = 34-36 teachers) 
Curriculum/content area instruction 
BPS's revised K0/K1 curriculum (Focus on K0/K1)— 83% Not surveyed on this topic 
Boston only 
Curriculum-specific training 98% 92% 
General content instruction (language/literacy/math 86% 82%, 
instruction) 
Alignment between content taught in K1 and later 60% 62% 
grades-Boston only 
Cognitively-demanding tasks 63% 68% 
Classroom environment and interactions 
Guiding and interacting with young children 91% 97% 
Language-rich interactions with children 91% 89% 
Understanding child development 81% 85% 
Classroom organization and learning environments 17% 82% 
Socio-emotional development 
Supporting children’s social/emotional development 91% 94% 
Behavior management 81% 82% 
Working with children with diverse needs 
Supporting learning of English Language Learners 82% 14% 
Supporting children with special needs in the 65% 71% 
Classroom 
Special education referrals 74% 68% 
Working with diverse populations 80% 80% 
Children/families with trauma 67% 62% 
Child and classroom assessment 
Training in using particular tools to understand the 809% 799 
classroom environment (ECERS, CLASS, etc.) ° ° 
Training in using particular assessment tools (i.e., ‘ 
ASQ, TS Gold, DIBELS) oe ep 
Conducting formative child assessments through P 6 

84% 97% 
observation, child screening, and/or assessments 
Working with families 
Communicating with parents 82% 80% 
Family engagement 84% 88% 
Other 
Health, safety, and nutrition 17% 94% 
Training on state standards 66% 1% 
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Exhibit 3.5: Usefulness of Training as Assessed by Lead and Non-Lead Teachers, 2016-17 


% Lead Teachers Who % Non-Lead Teachers Who 
Rated Training as “Very Rated Training as “Very 
Effective” Effective” 
(n = 42-46 teachers) (n = 34-36 teachers) 

Training Topic 
Curriculum/Content area instruction 
BPS's revised KO/K1 curriculum (Focus on K0/K1)— 60% Not surveyed on this topic 
Boston only 
Curriculum-specific training 47% 58% 
General content instruction (language/literacy/math 39% 57% 
instruction) 
Alignment between content taught in K1 and later 28% 43% 


grades-Boston only 
Cognitively-demanding tasks 37% 39% 
Classroom environment and interactions 


Classroom organization and learning environments 36% 46% 
Guiding and interacting with young children 50% 64% 
Understanding child development 49% 66% 
Language-rich interactions with children 48% 65% 
Children’s socioemotional development 

Supporting children’s social/emotional development 57% 69% 
Behavior management 32% 50% 
Working with children with diverse needs 

Supporting learning of English Language Learners 42% 64% 
eee children with special needs in the 29% 5% 
Working with diverse populations 46% 61% 
Special education referrals 25% 43% 
Child and classroom assessment 

Training in using particular tools to understand the 44%, 56% 
classroom environment (ECERS, CLASS, etc.) 

Hee aE ew assessment tools (i.¢., 38% 59% 
Conducting formative child assessments through , é 
observation, child screening, and/or assessments aos oe 
Working with families 

Communicating with parents 49% 61% 
Family engagement 47% 43% 
Other 

Health, safety, and nutrition 42% 59% 
Training on state standards 45% 50% 
Trauma training 38% 29% 


Notes. The proportion of teachers in each cell reflects those who responded that they received some training on a topic and indicated that the 
training was ‘very effective’ at helping improve their classroom practices. Response options included “very effective,’ ‘somewhat effective,’ 
‘neutral,’ “somewhat ineffective,’ and ‘somewhat ineffective.’ The proportion of lead teachers that reported each training topic to be ‘somewhat 
effective’ was also notable and ranged from 14 to 35 percent of those that received training. 
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2.2.3. Professional Development Adjustments Reported by Program Leadership 


The majority (65 percent) of the program directors and educational leaders interviewed across the five PEG 
communities reported making changes to the training offered to PEG teachers in Year 2 to improve its 
relevance and effectiveness. These changes involved increasing the overall amount of training and to the 
content covered. The most common reasons for these changes were to address needs identified by program 
staff (administrators and teachers) in Year 1 or in response to QRIS requirements (Exhibit 3.6). 


Exhibit 3.6: Reasons Directors Changed Focus of Professional Development Training, 2016-17 


# of 
Program # of 
Directors Communities 
Reasons Represented 


From the perspective of program leadership, the teacher training in Year 2 was primarily focused on three 
areas: 


1. Curriculum development: Nearly all program directors and education managers interviewed 
(13 of 14 respondents, representing all 5 communities) prioritized training that focused on 
improving their teachers’ understanding and implementation of the specific curriculum (and 
curriculum modules) being used in their PEG classrooms. Some of the training highlighted 
specific areas within the curriculum such as literacy and social emotional development, and other 
training focused on positive teacher-child interactions (as also emphasized by the CLASS 
observation protocol) in the context of the curriculum. 

2. PEG/QRIS requirements: Nearly 60 percent of program directors and education managers (8 
of 14 respondents, representing 4 communities) reported focusing teacher professional 
development training on achieving certain QRIS ratings that are expected as part of the PEG 
model. 

3. Supporting teachers as leaders: Just under half of program directors and education managers 
(7 of 14 respondents, representing 3 communities) reported establishing goals for developing the 
leadership skills of teachers. These program directors and education leaders defined leadership in 
different ways, from encouraging non-lead teachers to finish their bachelor’s degrees to creating 
opportunities for lead teachers to share their knowledge with other educators in their classrooms 
and beyond. 


Program directors were generally satisfied with how well the teacher training in Year 2 addressed teacher’s 
needs; their average rating of the relevance of the training was 3.86 on a 5-point scale. However, they also 
identified specific areas/topics in which PEG teachers could benefit from more professional development. 
Areas that were identified included: 


e Supporting children who have special needs (4 communities); 


e Dealing with challenging behaviors in the classroom (3 communities); 


Associates “Year 2 Massachusetts Preschool Expansion Grant (PG) Evaluation Report || pg. 14 


PROFESSIONAL LEARNING OPPORTUNITIES 


e Working with families (2 communities); 
e Literacy and language development (2 communities); and 


e Trauma (1 community). 


At the end of Year 1, teachers rated their professional development related to special education students as 
among the least effective and useful. Program directors in four of the communities also identified this topic 
as an area where more training is still needed. In Year 2, it does not seem that this training need has been 
fully addressed. 


2.3. Coaching for PEG Teachers in Year 2 


Information on coaching in Year 2 is based on information from interviews with PEG coaches from four of the 
five communities,’ supplemented by teacher perspectives from surveys. Comparisons between Years 1 and 2 
were not always possible because of a lack of comparable data. 


2.3.1. PEG Coach Characteristics 


In Year 2, in all five communities, school district employed coaches worked with the PEG teachers. Across the 
four communities included in this report, all but one PEG coach returned at the beginning of Year 2 (Exhibit 
3.7), while in the fifth community both coaches returned but left mid-year of Year 2. All PEG coaches had 
worked in the early education field before, either as a teacher, center or school leader, or specialist. Many PEG 
coaches had prior experience with coaching ranging from three to ten years. In one community, where a PEG 
coach had no prior experience with coaching, the district’s early childhood director provided close supervision 
of the coach that included field observations, after which the supervisor would provide targeted feedback about 
how the coach could improve her work with teachers. 


Exhibit 3.7: Number of PEG Coaches by Community, 2016-17 


# PEG coaches 
returning from Year 1 


# PEG coaches with 

coaching experience 

prior to PEG 

4 This community did not offer PEG coaching in Year 1. 

> Two returning coaches left mid-way through Year 2 and were replaced by new coaches. 


2.3.2 PEG Coach Job Responsibilities and Caseloads 


Across communities, each PEG coach had a caseload of 4 or 5 PEG classrooms (Exhibit 3.8), and in three 
communities, coaches also coached non-PEG classrooms. In addition to their direct coaching responsibilities, 
some coaches performed other non-coaching tasks in PEG programs and some held coaching and/or non- 
coaching roles in non-PEG classrooms. 


4 Coaches in the fifth community (Lowell) were not interviewed in Year 2 because the two coaches left the 


district during the middle of the year. New coaches were hired in the spring of 2017. 
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Exhibit 3.8: PEG Coach Caseloads and Other Responsibilities, 2016-17 


Springfield Holyoke 


Caseload (average # of 5 per coach 
Classrooms per coach) OU TCEE NT (1 coach provided 
coach had 2 : : 3-4 per coach 4 per coach 
supervisory coaching to 
Classrooms) 
all classrooms) 


Non-coaching and/or Curriculum NAEYC coaching, Coaching teachers | Professional development 


non-PEG development, overseeing PEG grant, in non-PEG for teachers, coaching in 
responsibilities NAEYC mentoring, family engagement, classrooms. PEG non- PEG classrooms 
professional professional development coaches were 
development for for teachers responsible for 
teachers, non-PEG between 2-10 non- 
coaching PEG classrooms 


2.3.3 Amount of Coaching Teachers Received 


Nearly all lead teachers (98 percent) and non-lead teachers (84 percent) reported receiving coaching from 
either the PEG district coaches or non-public school coaches in Year 2 (Exhibit 3.9). This represents an 
increase from the percentages reported in Year 1 (92% and 78, respectively). In both years, most coaching 
was delivered by PEG public school coaches. 


In Year 2, about one-fifth of lead teachers reported receiving more than 40 hours of public school coaching, 
and one-third each reported between 10 and 21 hours and fewer than 10 hours of coaching. The proportion 
of lead teachers reporting more than 40 hours of PEG coaching increased in Year 2. About one-third of lead 
teachers also reported receiving coaching from non-public school staff, although for most teachers, this 
constituted less than 10 hours of coaching during the year. 


Two-thirds of the non-lead teachers reported receiving coaching in Year 2. PEG communities made efforts 
to address the concern raised in Year 1 about more limited coaching for non-lead teachers, of note, in Year 
2, the majority of non-lead teachers received ten or fewer hours of coaching, still less than among lead 
teachers (Exhibit 3.9). 


Exhibit 3.9: Coaching Received by PEG Teachers, 2016-17 


Lead Non-Lead All 
Teachers Teachers Teachers 
(n = 45) (n = 37) (n= = 


[Any Coaching Recevedtiom..[a| % [W| % [N| % | 
[Non-pubieschoolcoastes ——=«d TS ae 6 | om [a] aoe 


PEG Public School Coaches: Number of Hours of Coaching 


a 
A 
EI 0 
Periedanous ————S~— | tm | me lm 
Foweradnouws——OSCSC=“‘*‘“~*~s CCC 
PDonatkow Sir | tm af om [el om | 
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There was variation in the amount of coaching by community (Appendix Exhibits A.2 and A.3). Overall 
teachers in two communities (Holyoke and Springfield) reported more PEG coaching than teachers in the 
other three communities (Boston, Lawrence, and Lowell). 


2.3.4 Focus of Coaching 


PEG coaches reported that in Year 2, the content of coaching sessions was more closely aligned to the 
professional development training than in Year 1. As reported by lead teachers, the most common focus of 
PEG coaching sessions was supporting instruction generally (i.e., not a particular content area), which is 
consistent with Year 1 (Exhibit 3.10). Supporting children's literacy and language skills was another 
common focus of coaching. Although over 60 percent of lead teachers reported receiving coaching in math 
instruction, less than half received coaching about science or social studies instruction. Fewer than half (41 
percent) of teachers reported receiving coaching on using assessments to guide instruction. There was some 
variation in the focus of coaching by community (Appendix Exhibits A.3 and A.4). 


Exhibit 3.10: Focus of PEG Coaching, 2016-17 


Lead Teachers Non- Lead All Teachers 
(n = 44) Teachers = 
a ae 
a SS es ees 
| Generalinstruction 


General instruction 


Supporting children's social/ emotional development 
Supporting children's play 
Classroom organization and learning environment 


Behavior management 
Using information from child formative assessment 
screeners and assessments 


Supporting English Language Learners 


a7 


Supporting children's social studies concepts 


na 
[Child development and classroom support | 
| 


2.3.5 Teacher Assessment of Coaching in Year 2 


Teachers had mixed opinions about the degree to which coaching had affected their behaviors (Exhibit 
3.11). All but four teachers reported that coaches had helped them improve in at least one area. Close to 
half of teachers (48 percent) strongly agreed that PEG coaching helped them to make better use of trainings. 
Fewer teachers strongly agreed that PEG coaching had helped them learn more about how to manage 
behaviors in the classroom (29 percent) or improved the way they support English language learners (25 
percent). Generally more teachers in Boston and Holyoke reported that they strongly agreed that particular 
components of coaching (such as being able to better use what they learned in trainings, creating better 
lesson plans, applying new skills in the classroom, and knowing how to effectively manage behavior) were 
helpful to them than teachers in Lowell and Springfield. 
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Exhibit 3.11: Teacher Perceptions of Helpfulness of PEG Coaching, 2016-17 


Lead Non-Lead All 
Teachers | Teachers Teachers 
(n = 44) (n= 31) (n = 75) 


| strongly agree that PEG coaching helped me because I... 


Have been able to better use what | learned in professional 4 48% 15 | 48% 36 | 48% 
development/formal group trainings. 


Famcreaingbeteressonpans.——S~Ss pm | 8 | oT | Oo 


Know much more about how to effectively manage behavior in the 
classroom. 


Am able to better support English Language Learners. | 9 | 20% | 10 | 32% | 19 | 25% | 


2.3.6 Coaching Process 


In interviews, coaches reported using a variety of approaches to individualize the topics and frequency of 
coaching sessions, including integrating teacher input. Coaches in three communities also reported using 
feedback from program directors to help design their coaching. Two communities used formal tools to 
guide decision-making around coaching topics. These tools captured curriculum implementation, 
instructional quality, and behavior management strategies used by teachers in the classroom. 


Coaching Sirategies 


In Year 2, PEG coaches reported the use of specific coaching strategies to guide their work. Together, these 
strategies indicate that coaches used a cycle that included observation and evidence collection, reflection, 
and feedback (Exhibit 3.12). 


Exhibit 3.12: Strategies Used by PEG Coaches, 2016-17 


# of 
Communities 


Strategy (n=4) 
Developing and tracking goals 

Observation of teacher in classroom 

Modeling/demonstration/role-play 

Reflection and feedback 

Training 

Co-teaching 


Relationship-building (time spent in initial stages of coaching to assess and 
understand classroom practices and needs) 


Providing resources (evidence-based practices, research articles, etc.) 


Formal observation measures and data collection 


Videos or recordings 
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While the coaches’ classroom visits generally involved a similar process (i.e. observation-debrief-goal 
setting), there was variability across communities in how coaching topics were selected: 


e In Boston, the focus of coaching sessions was driven by teacher needs based on teacher input and from 
a formal needs assessment conducted by coaches during a classroom observation at the beginning of 
the school year. The needs assessment focused on teachers’ behavior management, instructional 
strategies, and fidelity of curriculum implementation. Observation notes and a curriculum fidelity 
checklist were used by coaches throughout the year to refine the topics that they focused on with 
teachers. 


e In Springfield, shared coaching goals for the year were developed during meetings of all PEG lead 
teachers. These took the form of SMART goals (specific, measurable, achievable, results focused, and 
time bound). Coaches and PEG lead teachers reviewed videos of PEG classrooms during these meetings 
and discussed discrete steps that could be taken to meet the shared goals. Coaches also worked with 
individual classrooms on more targeted goals. Coaches reported that teachers videotaped themselves 
for use during self-reflection exercises. In addition to the feedback and direction provided by teachers, 
coaches used CLASS data and teacher reports of their own professional development goals to determine 
the focus of the individualized work. 


e In Lawrence, coaches reported that the focus of their work was driven by program leadership. These 
coaches reported that the overarching goals of the coaching work were decided at the beginning of the 
year during meetings of the program directors. The goals generally focused on QRIS requirements. 
Directors used CLASS data to determine the focus on asking open-ended questions, having feedback 
loops in the classroom, digging into thought processes behind child answers, and increasing student 
talk. Teachers were able to refine the focus of some coaching sessions by sharing additional information 
about their own needs. 

Documentation and Supervision of Coaching 

PEG coaches reported using more formal strategies to document their work with teachers in Year 2. All 

four communities reported using at least two methods for documenting the content of coaching sessions 

and the progress teachers made toward meeting goals. These methods included: 


e Observation notes (4 communities); 
e Contact logs (3 communities); 


e Formal observation tools such as curriculum fidelity checklists or the CLASS to document teacher 
progress (2 communities); and 


eA tracking system to help coaches assess whether the teacher had fully or partially met the goals set in 
a prior session (1 community). 


In three of the four communities, PEG coaches received formal supervision by district staff. In two of these 
communities, observation of coaches was part of the supervision. In all communities in which formal 
supervision was present, feedback was provided to coaches to improve their practice in the field. In one 
community, coaches created formal professional development plans at the beginning of the school year and 
were responsible for uploading evidence that they were making progress toward their goals. In another 
community, coach performance was evaluated using the state Educator Evaluation Framework that is used 
to regularly assess the performance of district educators. In addition, coaches in two of the four communities 
reported regular meetings with other district coaches to help support their work. 
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2.3.7 Coaching Successes and Challenges in Year 2 


Careful observation, honest feedback, modeling, and reflection were mentioned by program directors as 
key elements of effective coaching. Two program directors in one community also identified one-one-one 
meetings between coaches and teachers as highly important, while others pointed to the importance of 
directors being involved in the coaching process. 


Program directors identified several ways that they perceived teachers’ practice to have changed as a result 
of PEG coaching, including but not limited to: teachers taking leadership roles by sharing knowledge and 
perspectives with other teachers and center leadership (n= 3 representing 3 communities or 18 percent of 
directors interviewed); better implementation of the curriculum (n=6 representing 3 communities or 35 
percent of directors); more interactions during read aloud and more use of open-ended questions (n= 2 
representing 1 community or 12 percent of directors); and increased teacher confidence (n= 2 representing 
1 community or 12 percent of directors). When coaches were asked to describe the area or areas in which 
their teachers had improved the most in Year 2, they noted differentiation of instruction, using data to plan 
instruction, using PEG curriculum, creating positive classroom environments, and asking open-ended 
questions. 


While PEG coaches were largely positive about the work they had accomplished in Year 2, they also 
reported challenges. Coaches from the four communities struggled with limited availability of teachers for 
coaching sessions. In three communities, scheduling challenges resulted from inadequate teacher coverage, 
which interfered with the provision of timely feedback to teachers after observation sessions. In one 
community, coaches described the challenges of working with teachers whose schedules fluctuated; finding 
consistent time slots when both the lead teacher and the PEG coach were available was a struggle. Another 
community struggled with regular teacher absences that were not communicated in advance to the coaching 
staff. 


Program directors were more positive about staff coverage for coaching; a substantial number of directors 
(n=9 representing communities) reported sufficient staff coverage whereas others (n=3 representing 3 
communities) communicated having coverage “most of the time.” Overall, program director satisfaction 
with regards to coaching received was: 3.82 on a 5-point scale, with 5 being most satisfied. 


In one community, coaches struggled with effectively handing over control of the instructional leadership 
to program directors. While coaches developed strong relationships with some teachers, some coaches 
worried about the unexpected effect of making program directors too reliant on the coach when it came to 
working with teachers to improve their practice. One coach described this issue as interfering with true 
change because she did not think that the coach had the authority to make changes in response to barriers 
the teachers described in coaching sessions. In Year 2, this community had begun providing PEG coaching 
to program directors in order to address this issue, but encountered issues with the directors’ capacity to 
add another responsibility to the multitude of other duties that they were already responsible for. 


2.4 Paid Release Time for Instructional Planning in Year 2 
2.4.1. Amount of Release Time 


Although the PEG model does not require teachers to have formal release time to spend away from the 
classroom to plan for instruction, programs are encouraged to provide this time. In Year 1, almost 60 percent 
of teachers reported receiving at least a few hours per week of formal release time and in Year 2, this 
proportion increased to 76 percent of teachers (Exhibit 3.13). About one third of lead and non-lead teachers 
reported receiving release time once per week (Exhibit 3.14). The amount of paid release time reported by 
teachers was similar across communities (Appendix Exhibits A.4 and A.5). 
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Exhibit 3.13: Hours of Paid Release Time for PEG Teachers, 2015-16 and 2016-17 


None 15 30% 7 | 15% 9 29% 10 26% 


>0-1 hour/week Not asked 2 4 Not asked 2 5 
1-5 hours/week 20 53 30 65 18 58 22 58 
6-10 hours/week 1 3 5 11 dl 3 1 3 
11-20 hours/week 2 5 0 0 0 0 1 3 
Don't Know 0 0 0 2 6 1 3 
Other 0 0 2 1 3 1 3 


Exhibit 3.14: Frequency of Teacher Paid Release Time, 2016-17 


1 day/week 12 32% 10 36% 
2 days/week 4 11 3 11 
3 days/week 4 11 1 4 
4 days/week 1 3 0 0 
Every day 4 11 5 18 
Don't Know 2 5 5 18 
Other 11 29 4 14 


More than half of PEG lead teachers (55 percent) reported that they did not have enough release time to 
complete their job responsibilities, which is slightly higher than what was reported in Year 1 (Exhibit 3.15). 


Exhibit 3.15: Teacher Perception of Sufficiency of Amount of Paid Release Time, 2015-16 and 
2016-17 


[ecm elsl ILE) || ci 50% | 22| 55% 4 | 19% 6 21% 
complete my responsibilities 
| have just enough time to 8 36% 44 35% 16 76% 16 55% 


complete my responsibilities 


| have more than enough time 
to complete my 3 14% 4 10% 1 5% if 24% 
responsibilities 


Program directors described several strategies that they used to support productive teacher planning time 
including: formal or informal teacher planning teams (n=5 representing 5 communities); monthly lead 
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teacher meetings (n=4 representing 2 communities); and reliance on coaches to work with individual 
teachers on curriculum planning (n= 2 representing 2 communities). In one community, several program 
directors did not report any specific strategies to support productive teacher planning time. 


2.5 Teacher Collaboration 


All PEG lead teachers and nearly all non-lead teachers reported collaborating professionally in some way 
with other teachers in Year 2 (Exhibit 3.16). About half of the teachers (both lead and non-lead) reported 
some peer coaching. Nearly half of the lead teachers (46 percent) and one-third (34 percent) of the non-lead 
teachers were part of a formal collaborative network. 


Exhibit 3.16: Types of Professional Collaboration among PEG Teachers, 2016-17 


Lead Teachers Non-Lead Teachers All Teachers 
=a) , * ay ae =a <a 


} Meetings with teachers from other classrooms with teachers from other classrooms ate tae 


a conversations with teachers from other a ee 62% 
classrooms 


Peer |Peercoaching = | 52%] 19 | | 50% | 43 | 


oe network of teachers/professional learning +" _ et 40% 
community 


Peer observation and feedback ——— | 47% | — 


a 


Among lead teachers, the most frequently reported focus of the formal collaboration was instructional 
planning and planning instruction to improve students’ academic growth (Exhibit 3.17). Implementation 
of the overall PEG model and improving teacher performance were also frequent focuses of collaboration. 
Making sense of assessment data was less common; it was reported by less than half of lead teachers (40 
percent) and one-quarter of non-lead teachers. 


Exhibit 3.17: Focus of Formal Teacher Network among Participating PEG Teachers, 2016-17 


Lead Teachers Non Lead Teachers All Teachers 
ay Te ay eal ne =32) 


Cd 
TinpeneninghePeESmoat om] aL oH] 
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a diverse learners (children with IEPs 40% 50% 14 44% 
and English Language Learners) 


‘Abt Associates "Year 2 Massachusetts Preschool Expansion Grant (PG) Evaluation Report | pg. 22 


PROFESSIONAL LEARNING OPPORTUNITIES 


3. Teacher Outcomes: Job Satisfaction and Retention 


3.1 Teacher Job Satisfaction 


Providing PEG teachers with salaries on par with local public school teachers is an important component 
of the PEG model and represents a large proportion of program costs. This section of the report describes 
teacher job satisfaction, including satisfaction with their salaries and teacher turnover. In a broad sense, 
teachers felt more satisfied with different aspects of their job in Year 2 compared to Year 1, especially in 
terms of their ability to teach in the program (Exhibit 4.1). 


Exhibit 4.1: Comparison of Teacher Job Satisfaction from 2015-16 versus 2016-17 


Lead Non-Lead 
Teachers Teachers 
(n = 41) (n=27) 


Compared to Year 1, in Year 2 | agree that | feel... 


(er nce ad 


More suppodedeubped io del wih sudentardnessatweneeim | v8 | am [9 | 3% 


In Year 2, 23 percent of lead teachers strongly agreed and 42 percent somewhat agreed that they were 
satisfied with their salary, and most lead teachers (60 percent) strongly agreed with the statement that they 
deserved more than they are currently paid. These reports are very similar to those reported in Year 1. Of 
note, only 5 percent of non-lead teachers strongly agreed and 54 percent somewhat agreed that they were 
satisfied with their salary. 


Almost all teachers were satisfied with student readiness for kindergarten and progress made during 
preschool. About three-quarters of PEG lead teachers were satisfied with having a coach and two-thirds 
were satisfied with their training. 


Exhibit 4.2: Teacher Satisfaction with Aspects of the PEG Program Viewed as Important, 2016-17 


Non-Lead 
Lead Teachers Teachers 
(n = 44) (n = 33) 

lam satisfied with... n % n % 
Your student readiness for kindergarten/progress made 42 95% 29 88% 
Classroom curriculum 37 84% 26 19% 
Knowing what is expected of you 37 84% 24 73% 
Having a coach 33 15% 24 13% 
Useful, interesting professional development 28 64% 27 82% 
Your salary 22 50% 16 48% 


Note: The items included are the six aspects of their jobs that over 80 percent of teachers rated as important. 


Teachers were mixed about the supportiveness of their program leadership in helping them to be effective 
educators. About one-third (37 percent) of lead teacher reported that program leadership was very 
supportive, yet one-half of lead teachers reported that program leadership was either somewhat supportive 
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or neutral. Non-lead teachers were more negative, with 18 percent of non-lead teachers reporting that they 
felt program leadership was somewhat unsupportive of supporting them to effectively teach. 


3.1.1. Correlates of Teacher Satisfaction 


Exploratory analyses were also conducted to look at whether there was a relationship between teachers’ job 
satisfaction and the level or amount of professional learning opportunities teachers experienced in Year 2. 
An overall measure of teacher job satisfaction was calculated by averaging teachers’ ratings across six 
program areas. Overall job satisfaction was correlated with each of the three types of professional supports 
(amount of training, coaching, and paid release time) to examine whether teachers with more professional 
opportunities had higher job satisfaction. 


There was mixed evidence about the relationship of job satisfaction and levels of professional supports, 
with generally positive trends but also inconsistencies. For training, teachers who reported the most training 
had higher average job satisfaction compared to teachers with moderate and low levels of training (Exhibit 
4.3). However, the small group of teachers who reported the fewest hours of training also had similarly 
high job satisfaction, which does not follow the general upward trend. 


For coaching, the teachers with the least and the most hours of coaching had higher satisfaction ratings 
(Exhibit 4.4). The relationship of satisfaction to frequency of coaching showed a more consistent upward 
trend in job satisfaction (Exhibit 4.5). 


For hours of paid release time, the teachers with higher levels of paid release time (6 or more hours a week) 
reported higher job satisfaction compare to teachers with fewer hours (Exhibit 4.6). There was no difference 
in job satisfaction among teachers who had a few hours of release time and those with no release time. 


Abt Associates Year 2 Massachusetts Preschool Expansion Grant (PG) Evaluation Report | pg. 24 


PROFESSIONAL LEARNING OPPORTUNITIES 


Exhibit 4.3: Teacher Job Satisfaction by Amount of Training Received, 2016-17 


100% 
° 92% 92% 


80% 74% 73% 
66% 
60% 
40% 
20% 
0% a — ——_—. —__. oe 


0-5 hours (n=4) 6-10 hours (n=8) 11-20 hours (n=9) = 21-40 hours (n=14) More than 40 hours 


Percent of Different Aspects Satisfied With 


(n=8) 
Amount of Training Reported 
Exhibit 4.4: Teacher Job Satisfaction by Amount of Coaching Received, 2016-17 
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Exhibit 4.5: Teacher Job Satisfaction by Frequency of Coaching Received, 2016-17 


100% 
89% 


83% 
80% 
68% 
60% 
40% 
20% 
0% ———— —— — 


One or two times a year (n=13) Monthly (n=8) Weekly (n=11) 


Percent of Different Aspects Satisfied With 


Frequency of Coaching Reported 


Exhibit 4.6: Teacher Job Satisfaction by Amount of Paid Release Time, 2016-17 


100% 


87% 


80% 78% 78% 
60% 
40% 
20% 
0% ———— 


None ee 7) 1-5 ee (n=29) 6-10 hours/week (n=5) 
Amount of Planning Time Reported 


Percent of Different Aspects Satisfied With 


3.2 Teacher Turnover 


PEG programs retained most of their teaching staff in Year 2. The substantial majority of lead teachers (77 
percent) and non-lead teachers (70 percent) continued teaching in the second year of the PEG program. 
Program directors reported during interviews that salaries—and the uncertain status of funding for the PEG 
classrooms in the future—were key drivers of teacher turnover, and believed this was mitigated only by 
existing relationships with program staff and occasional fringe benefits not reflected in salary levels. 
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Program directors also noted that a number of teachers who left the program in Year 2 left for higher-paying 
teaching jobs versus leaving the field for another sector. 


Program directors reported that new teachers were introduced to the PEG model during orientation and 
familiarized with the key supports that each program was expected to provide. Staff turnover data was not 
tracked directly by programs, but program directors interviewed in Year 2 frequently reported that teacher 
turnover outpaced the program directors’ ability to hire replacement teachers (especially for assistant 
teachers). Directors remarked that many of the same “push” factors that drove lead teacher turnover (e.g. 
salary, benefits) also drove turnover among the assistant teaching staff. Yet non-lead teachers reportedly 
did not experience some of the “pull” factors (e.g., access to extensive professional supports) that kept lead 
teachers at the programs. In Year 2, PEG programs sought to change this by offering more professional 
development and coaching to assistant teachers. 


Overall, program directors’ satisfaction with being able to hire the qualified staff they hoped for was 3.44 
on a 5-point scale, where 5 indicated most satisfied. The level of involvement that program directors had 
in recruitment and hiring of staff varied between programs. Five of the program directors interviewed 
identified their agency’s Human Resources (HR) department as the primary group responsible for hiring, 
recruitment, and retention processes. Three of these program directors interviewed potential candidates 
after HR provided them with the contact information. The remaining four directors reported relying on 
assistants, the district, or a supervisor for hiring, recruitment, and retention. 
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4. Classroom Quality Outcomes 


In the winter of 2017 (February-March), data collectors conducted an observation of each PEG classroom 
using the Classroom Assessment Scoring System (CLASS) and the Early Language and Literacy Classroom 
Observation (ELLCO). The observation of each classroom began when the school day started and continued 
until children started naptime (typically between 8 am and 1| pm). 


4.1 Findings from the CLASS Observations 


The CLASS measures overall instructional quality with a focus on interactions among teachers and students 
in the classroom. The CLASS produces four scores: an overall score and a score for each of three domains. 
Each score ranges from | to 7. A score of 1-2 is described as “low” quality in that aspect of teacher-child 
interaction. Scores of 3-5 are described as “moderate,” and scores of 6-7 are described as “high” quality. 


e Total Score: The total score combines the three domains. 


e Instructional Support Domain: How well the teachers use instruction to help children learn. The domain 
has 3 dimensions: Concept Development, Quality of Feedback, and Language Modeling. 


e Emotional Support Domain: How well teachers promote a positive climate through their interactions. The domain 
has 4 dimensions: Positive Climate, Negative Climate, Teacher Sensitivity, and Regard for Student Perspectives. 


e Classroom Organization Domain: How well the classroom is organized to facilitate learning. The 
domain includes 3 dimensions: Behavior Management, Productivity, and Instructional Learning 
Formats. 


4.1.1. Year 2 CLASS Findings 


In Year 2, the total CLASS average score across all PEG classrooms was 4.9. The average scores statewide 
for two of the CLASS domains—Emotional Support and Classroom Organization—teflected a level of 
quality that was close to “high” as defined by the developers of the measure (scores of 5.9 and 5.7, 
respectively). The average score for Instructional Support also reflected moderate quality; although the 
score was substantially lower than the other two domains at 3.1, it compares favorably to other national 
samples of programs. Generally, scores tend to be lower on the instructional support domain compared to 
the other two, and scores above a 3.25 on instructional support have been found to have a positive 
relationship with child outcomes (Burchinal et al., 2010). 


The average CLASS scores for the individual communities showed a similar pattern, with higher scores for 
Emotional Support and Classroom Organization than for Instructional Support (Exhibit 5.1). The average 
scores for the communities did not vary much for Emotional Support (averages ranging from 5.8 to 6.2); 
the variation in scores for Classroom Organization was greater (averages ranging 5.3 to 6.0). For 
Instructional Support, scores ranged from 2.9 to 3.5. 
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Exhibit 5.1: CLASS Average Total and Domain Scores by PEG Community and PEG Statewide, 
Winter 2017 (n = 48 classrooms® 


Total Score 
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4 Boston=15, Holyoke=4, Lawrence=10, Lowell=8, Springfield=11 


4.1.1 Comparison of Change in CLASS Ratings from Year 1 to Year 2 
The CLASS scores improved from Year | to Year 2 (Exhibit 5.2). 


e For Classroom Organization, the average CLASS score across all PEG classrooms rose from 5.2 to 5.7. 
Substantially more classrooms were rated as high quality on this domain in Year 2 (75 percent) than in 
Year 1 (40 percent). In addition, the average Year 2 score overall was significantly higher than the 
average Year | score. 


e For Emotional Support, the Year 2 score was more similar to the Year 1 score (5.9 vs. 5.7), but the 
proportion of classes rated as high quality increased from 60 percent to 80 percent. The average Year 
2 score was significantly higher than the average Year | score. 

e For Instructional Support, though the proportion of classes rated as moderate quality increased slightly 
from 75 percent to 79 percent, the average score remained relatively unchanged from 2016 to 2017. 
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Exhibit 5.2: Percentage of PEG Classrooms in Different Quality Categories by CLASS Domain, 
Winter 2017 and Winter 2016, All Classrooms (n = 48 classrooms) 


Classroom Organization - 2017 


Avg=5.7 out of 7 
Avg=5.2 out of 7 


Classroom Organization - 2016 


Emotional Support - 2017 Avg=5.9 out of 7 


Avg=5.7 out of 7 


Emotional Support - 2016 


Low (1-2) ™Moderate (3-5) = High (6-7) 


Instructional Support - 2016 23% 


Note: Although the overall average score for all 48 teachers on instructional support decreased just slightly from Year 1 to Year 2, it was not a 
significant change, and a higher percentage of classrooms were in the “moderate” quality category and fewer were in the “low” category in Year 
2, suggesting some overall improvement. Because the average instructional support scores within the “moderate” and “high” categories were 
slightly lower in Year 2 than in Year 1, the overall average across quality categories in Year 2, despite the upward shift of classrooms within 
categories. 


Improved scores from Year | to Year 2 may reflect development over time in teachers who were there for 
both years. The majority of lead teachers remained from Year | to Year 2, and excluding new teachers from 
the analysis, the average scores improved on all three domains (Exhibit 5.3). 


Exhibit 5.3: Percentage of PEG Classrooms in Different Quality Categories by CLASS Domain, 
Winter 2017 and Winter 2016, Classrooms with Same Teacher in 2016 and 2017 (n = 
36 classrooms?) 


Classroom Organization - 2017 


Avg=5.8 out of 7 
Avg=5.3 out of 7 


Classroom Organization - 2016 


Emotional Support - 2017 Avg=6.0 out of 7 


Emotional Support - 2016 Avg=9.7 out of 7 


Instructional Support - 2017 
Avg=3.2 out of 7 


Instructional Support - 2016 Avg=3.2 out of 7 


Low (1-2) Moderate (3-5) High (6-7) 
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4.2 Findings from the ELLCO Observations 


The Early Language and Literacy Classroom Observation (ELLCO) measures the quality of support for 
language and literacy development, including support for diversity of languages, abilities, and cultures. The 
ELLCO produces three scores: an overall score and a score for each of two subscales. Each score ranges 
from 1 to 5. A score of 1 is described as “deficient” evidence of quality. A score of 2 is described as 
“inadequate,” a score of 3 is described as “basic,” a score of 4 is described as “strong,” and a score of 5 is 
described as “exemplary.” 


e Total Score: The total score combines the two subscales. 


e General Classroom Environment Subscale: How well the classroom is organized and promotes general 
learning. The subscale has 2 domains: Classroom Structure and Curriculum. 


e Language and Literacy Subscale: How teachers and students engage in literacy and language activities. 
The subscale has 3 domains: Language Environment, Books and Book Reading, and Print and Early 
Writing. 


4.2.1. Year 2 ELLCO Findings 


The total average Year 2 ELLCO score across PEG classrooms was 3.7. The average score for the General 
Classroom Environment subscale was 4.0, which represents strong level of quality. The average score for 
the Language and Literacy subscale was lower—3.5, representing a “basic” level of quality. 


The five PEG communities did not vary widely on their average scores for General Classroom Environment 
(Exhibit 5.4). The average scores across the communities ranged from 3.9 to 4.3. There was more variation 
in the community-level scores for Language and Literacy, for which the average scores ranged from 3.3 to 
3.8. 


Exhibit 5.4: ELLCO Average Total and Subscale Scores by PEG Community and PEG Statewide, 
Winter 2017 (n = 48 classrooms?) 
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4.2.2. Comparison of Change in ELLCO Scores from Year 1 to Year 2 


The average scores at the state level did not change from Year 1 to Year 2. Across years, for teachers who 


remained PEG teachers both years, most teachers were rated at the same level of quality in each year 
(Exhibit 5.5). 


Exhibit 5.5 Percentage of PEG Classrooms in Different Quality Categories by ELLCO Subscale, 
Winter 2017 and Winter 2016, Classrooms with Same Teacher in 2016 and 2017 (n = 36 
classrooms) 
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5. Findings on Family Engagement and Comprehensive Services 


This chapter examines the family engagement activities offered during Year 2 of PEG implementation, 
based on information from interviews with family support specialists and program administrators and 
teacher surveys. In Year 2, parents were not asked directly about their involvement with PEG. Changes 
observed between Years 1 and 2 are highlighted, with a focus on Year 1 implementation challenges that 
were successfully addressed and challenges that persisted or were newly identified in Year 2. 


5.1. Family Engagement Activities 


Family engagement activities may be particularly important for PEG families because this program is often 
the family’s first experience with formal early education. The family engagement component of the PEG 
model reflects the importance of the family in the child’s education and learning, beginning in preschool 
and continuing into the child’s school years. Parent engagement efforts are intended to help parents 
understand what and how their child is learning, as well as to help communicate ways that parents can 
support and extend classroom learning at home. 


Effective family engagement programs are generally described as having at least a few of the following 
features: (a) individual contacts between the child’s classroom teacher and the family, (b) multiple 
opportunities for parents to contribute to the program so as to appeal to parents with different levels of 
comfort and interest, (c) activities at the beginning of the year that introduce and orient families to the 
program and make them feel welcome and comfortable, (d) family events that are responsive to families’ 
interests and needs, and (e) systems for tracking the engagement of the families so as to be able to reach 
out to families who are not engaging with the program (Sharpe, Davis, & Howard, 2017). 


In Year 1, programs reported that their family engagement programming had not reached full 
implementation by the end of the year and noted that parent attendance was an issue. Although programs 
offered a variety of ways for parents to learn about and become involved in PEG, programs did not have 
systematic tracking systems in place that allowed them to document levels of family participation in 
different types of activities. 


5.1.1. Gains in Implementation of Family Support in Year 2 


Overall, family engagement was perceived as an area of improvement by a majority (65 percent) of program 
directors in Year 2. These program directors reported that they provided more responsive services and 
programs to families in Year 2 as compared to Year 1. 


The following positive changes were reported for PEG family support efforts during Year 2: 


e Increased number of programs with dedicated family support staff; 

e Increased coordination in family support services across programs in the same community; 

e Increased consistency of individual contact with families through parent-teacher conferences with all 
families; 

e New strategies for connecting with families that built connections among parents as well as linking 
parents to their child’s learning and development in the classroom; 


e Family events with less focus on holidays and celebrations and increased focus on educating parents 
about their children’s classroom curriculum content and goals and parent needs; and 


e Enhanced use of partnerships with local agencies and charities for parent education classes. 
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5.1.2 Additional Programs with Dedicated Family Support Staff 


In Year 2, 11 of 16 ELPs employed family support specialists to help promote and provide education and 
engagement opportunities as well as connect families to needed supports. Family support specialists 
reported spending between 10 and 25 percent of their time on tasks other than family engagement activities 
and comprehensive services coordination (Exhibit 6.1). These tasks included: tracking attendance and 
reaching out to families whose children were absent a great deal, providing classroom behavior support, 
tracking child and family data, crisis management, working with children with special needs, facilitating 
team meetings, supporting teachers, observing children in the classroom, and assisting with enrollment. 


Exhibit 6.1 Family Support Specialists’ Use of Their Time in Year 2 


Community % of Time on Family % of Time on % of Time on Other 
Engagement Comprehensive Job Responsibilities 
Services 


10-25% 25-75% 10-50% 


Holyoke 
Springfield 
Lawrence 


Note. Information about the breakdown of family support staff time in Lowell was not available during interviews. 


5.1.3 Increased Community Coordination of Family Support Activities 


In Year 2, there appears to have been more consistent emphasis on family supports and more coordinated 
community-level programming. In four communities, family support specialists operated at a community- 
wide level even though they were employed by a specific ELP; they coordinated efforts across ELPs to 
provide consistent family and community engagement activities. In the fifth community, there was less 
coordination across ELPs, although monthly PEG meetings were reportedly an avenue for discussions of 
availability of resources across the community. 


5.1.4 Individual Communication with Families 


Increased numbers of parent-teacher conferences. In Year 2, all PEG programs conducted at least one 
parent-teacher conference over the course of the school year, which was a small increase over Year 1. In 
both years, the majority of teachers reported conducting several conferences per year, and a small number 
of teachers (11 percent of lead teachers and 19 percent of non-lead teachers) reported holding monthly 
conferences. 


Similar frequencies of home visits. Family support specialists from all five communities reported that all 
PEG families in their programs received at least one home visit in Year 2.° Family support specialists 
interviewed from Lowell and the programs in Boston that had staff in this role reported that all families 
received at least two home visits. The first home visit, which was typically conducted near the beginning 
of the year, focused on getting to know parents and the child’s home environment and understanding 
parents’ expectations for their child and for the PEG program. When a second visit occurred, it often 
focused on the PEG child’s learning progress and included follow-up about any needs and questions 
discussed during the first visit. 


Family support specialists from all five communities reported conducting home visits with the classroom 
teacher. The teachers often led the meeting and shared detailed information about the child and his or her 


> Not all PEG programs had dedicated family support specialists. 
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behavior and learning in the classroom with parents. The family support specialists were typically 
responsible for exploring family needs and strengths during the visits. 


In Year 2, teachers from 18 of 26 programs reported home visits, and about 70 percent of teachers 
reported at least one visit per year. Of the 18 programs, 17 were from ELPs that also employed family 
support staff. In the programs where home visits were conducted, about 40 percent of teachers reported 
one home visit with each family, and one-third conducted several homes visits to each family. About half 
of teachers reported that the family support specialists accompanied the classroom teacher on the visits 
(Exhibit 6.2). There were multiple focuses to the home visits including relationship building, learning 
about parent expectations and the home environment, and finding out about the needs of families (Exhibit 
6.3). 


Exhibit 6.2: Frequency of Home Visits as Reported by Lead Teachers, 2016-17 


State Boston Holyoke Lawrence Lowell Springfield 
ae al a ea =e cs Evel a Bua En I al 
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Exhibit 6.3: Participants in Home Visits as Reported by Lead Teachers Who Conducted Visits, 
2016-17 
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Exhibit 6.3: Focus of Home Visits as Reported by Lead Teachers Who Conducted Visits, 2016-17 
State Boston Holyoke | Lawrence Lowell Springfield 
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Other strategies for ongoing communications with families. In Year 2, programs continued to use parent 
drop-off and pick-up as an opportunity to connect with parents and build relationships. In Year 2, four of 
the five communities reported sending out a monthly newsletter with classroom and program news and 
parenting hints. One community reported supplementing the newsletter with a photo sharing application 
that teachers used to connect families to the daily activities of their children. Another community 
implemented a new strategy for connecting with parents by holding monthly parent coffee house meetings 
during which parents were invited to look at what was happening in their children’s classrooms. 


5.1.5 Enhanced Programming for Family Events 


In Year 2, programs continued to offer a variety of center-wide parent/family events (Exhibit 6.4). The 
content of the programming changed in Year 2, moving away from more traditional holiday-themed 
celebrations to more focused programming intended to increase parents’ capacity to support their child’s 
learning. New events focused on parent involvement in their child’s learning and development, such as: 


e Aseries of events to increase fathers’ involvement; 


e Events focused on what the children were learning in the classroom, including an event based on the 
PEG literacy curriculum and an event called “STEM to STEAM” focused on science, technology, 
engineering, art, and math games that could be played at home; 

e Aworkshop to help parents integrate painting activities into their play with their children; and 


e Asummer safety event that included the larger community. 


A second new focus of family events was addressing parent needs. One program held a resource fair to 
connect families to social services, employment, and GED information, and organized a series of workshops 
focused on resume writing, job searches, and interviewing skills. 
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Exhibit 6.4: Family Engagement Activities Reported by Community, 2016-17* 


ee Holyoke 


Orientation to PEG 
kindergarten 
celebrations 


PEG classroom 
curriculum, 
activities 


Parent education/ 
employment skills 


Parenting 


PEG literacy 
curriculum (in 
collaboration with 
public school district) 
(collaborated with 
Holyoke) 


STEM to STEAM: 
science, technology, 
engineering, art, and 
math games at home 
(collaborated with 
Holyoke) 


Classes through a 
parent café format 
responding to 
interests of parents 


Fathers’ involvement 
(collaborated with 
Holyoke) 


Integrating art 
activities in play at 
home with child 


Literacy activities in 
the home 

Summer safety event 
(collaborated with 
Holyoke) 


STEM to STEAM 
events 
(collaborated with 
Springfield) 


PEG curriculum 


Classes on ESL 
instruction, tax 
preparation 


Parent workshops 
on resume writing, 
job searches, 
interview skills 


Resource fair for 
parents about 
social services, 
employment and 
GED opportunities 


Fathers’ 
involvement 
(collaborated with 
Springfield) 


Summer safety 
event 
(collaborated with 
Springfield) 


v 


Citizenship/immigr 
ation resource 
night (community- 
wide) 


Handouts and 
brochures about 
community 
services 


Range of classes 
including 
computer, ESL 
instruction through 
partnership with 
local charity 


Parent education 
class on strategies 
for dealing with 
challenging 
behaviors 


[are a a a a 


Monthly parent 
coffee house 
meetings where 
parents observe 
children’s 
classrooms 


Parent classes/ 
workshops on 
financial literacy 


Educational 
materials 
provided to 
parents via a 
resource table. 


GED classes 
and access to 
resources 
through a 
community 
partner. 


Note. The activities listed reflect information gathered via interviews with select program directors and family support staff, and therefore likely 
are not a comprehensive list of the activities offered in each community. 
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5.1.6 Enhanced Parent Education through Community Partnerships 


In Year 2, some programs expanded their programming for parents through partnerships with community 
partners, including local charities that were already providing parent classes in the community on work- 
related topics such as building computer skills, writing resumes, and job search strategies. 


5.1.7 Parent Classroom Volunteers Stayed at About the Same Level in Year 2 


In Year 2, programs continued to offer parents opportunities to volunteer in the classroom, with varying 
degrees of success. About half of the teachers reported having had parent volunteers in classroom events 
on a regular basis, nearly identical to the proportion of teachers who reported the same in Year 1. In both 
Year 1 and Year 2, most teachers who had parent volunteers reported that it happened irregularly, i.e., a 
few times during the year; only 14 percent reported having a parent volunteer in the classroom at least once 
each month. In Year 2, half of teachers reported that fewer than 50 percent of families volunteered at some 
point, and only about 25 percent of teachers reported engaging 75 percent or more of the families in the 
classroom. 


5.1.8 Outreach to Families 


To remain connected to and supportive of families, family support specialists communicated with parents 
using a range of strategies, including flyers, e-mails, texts, newsletters, phone calls, and face-to-face 
conversations. Family support specialists used different strategies to recruit current families to attend 
engagement events and to communicate about available comprehensive services. These strategies included 
both word-of-mouth and face-to-face communication. In one community, the family support specialist 
reported making calls to parents in advance of the events to increase participation. In another community, 
the teachers placed brochures about events in children’s backpacks and gave the children reminder bracelets 
the day before or the day of family engagement events. 


Family support specialists in four out of five communities reported that face-to-face conversations with 
parents were the most successful strategy they or other staff in their program used for communicating about 
and recruiting participation in family engagement activities and comprehensive services. These family 
support specialists felt that the relationships between center staff and the parents increased parents’ 
commitment to participate. In the fifth community, the family support specialist discussed the importance 
and success of increasing her availability to the families so that parents would feel comfortable and would 
reach out when needed 


5.1.9 Ongoing Challenges 


Although PEG program staff believed that the family support activities were more organized and systematic 
in Year 2, this change did not seem to translate into large increases in the level of family participation. A 
substantial majority of lead teachers (68 percent) reported that half or less than half of families engaged in 
program activities. 


Teachers also reported the degree to which six different challenges were present in the population of 
families with whom they worked (Exhibit 6.5). In general, similar percentages of lead teachers in Year 1 
and Year 2 perceived these issues as presenting challenges, although a smaller proportion of lead teachers 
reported that lack of parental involvement was a challenge in Year 2 (9 percent versus 19 percent). About 
one-fifth of teachers reported that absenteeism and tardiness were serious issues in Year 2. 
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Exhibit 6.5: Lead Teacher Perceptions of Serious Problems Affecting PEG Children, 2015-16 and 


2016-17 
Lead Teachers 

Year 1 Year 2 

(n = 39) (n = 46) 
n (%) n (%) 

Family Challenge | % | 

Lack of parental involvement 6 19% 4 9% 
Poor student health 2 6% 3 1% 
Poverty 3 10% 4 9% 
Student absenteeism u 23% 9 20% 
Student tardiness 5 16% 10 22% 
Students unprepared to learn 3 9% 4 9% 
Teacher absenteeism 1 3% 2 4% 


In both Year 1 and Year 2, program administrators and family support specialists reported that the primary 
challenge with engagement was families not having time to attend events because of conflicting work and 
family responsibilities. Two programs reported budget constraints as a challenge for providing families 
with food at events; one program reported that funding from a local charitable organization was the only 
way that it could provide food to families. One family support specialist reported that the geographic 
accessibility of the events affected attendance, with 75 percent of families attending events at the center 
near the neighborhood in which many families lived and between 25 and 50 percent attending the less 
accessible location. 


PEG programs did not report using formal systems for tracking communications with parents and parent 
participation in family activities. The lack of a central tracking system made it difficult for them to provide 
accurate estimates of levels of family involvement across different types of activities. 


In most of the PEG programs, family support specialists wore multiple hats in terms of their responsibilities. 
Family support specialists played a leadership role in organizing and delivering the comprehensive services 
to families. Some programs also expected for the specialists to perform administrative tasks in the center 
where they worked and to provide coverage in classrooms. Family support specialists were often asked to 
participate in recruiting and enrolling new families. Other specialists operated more like social workers and 
were primarily involved with tasks related to the provision of comprehensive services to families, including 
assessing family needs, linking families with resources, tracking families’ receipt of services, and follow- 
up with families as needed. Some specialists also provided support directly to teachers by observing and 
providing feedback. 


5.2 Comprehensive Services 


In Year 2, as in Year 1, PEG offered comprehensive services to families through a combination of program, 
district-, and referral-based staff. Family support specialists reported being more intentional and proactive 
in their planning for comprehensive services in Year 2. These specialists reported that they had stronger 
relationships with the parents and the community, which translated into better services for the families. It 
appeared that in Year 2, PEG programs more consistently linked parents and children to a range of health 
and social services and also provided more services on-site, especially mental health support services. 
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Programs also used external mental health agencies and consultants to work directly with families on ways 
to reduce stress. One community focused specifically on counseling for families of children who had 
experienced trauma. In Year 2, PEG program staff also reported improved ability to link families to material 
supports, such as food and clothing and transportation. 


In general, family support specialists identified a high need for services among PEG families. The 
specialists identified a number of challenges faced by families, most directly related to their lack of income. 
PEG families were reported to frequently face unemployment, housing instability, and transportation 
challenges. In two communities, specialists reported that increased numbers of families were facing 
homelessness or having to double up with other family members. Many PEG families lived in 
neighborhoods with high levels of violence. Some PEG families struggled with mental health and physical 
health problems. In two of the communities, family support specialists described families as having 
increased fears about deportation, which both negatively impacted the families’ willingness to get involved 
with the PEG program and increased the likelihood of the families disappearing suddenly. 


5.2.1. Assessing Family Needs 


In Year 2, family support specialists in each community reported using both informal and formal methods 
to assess the needs of families. The informal strategies included conversations at drop-off and pick up, 
phone calls to families, and conversations that happened during family engagement or community events. 
Family support specialists reported that these methods provided information about needs that may not have 
been known at the beginning of the school year. Family support specialists also mentioned that as 
relationships were built through these informal contacts, parents felt more comfortable sharing information 
about their needs. 


Formal strategies used to assess needs included intake or enrollment surveys, parent-teacher meetings, and 
home visits. For example, in one community the process of assessing family needs began during the 
enrollment appointment, when parents’ concerns about their children were recorded. The family support 
specialist in this community followed up with families about these concerns by phone or during a home 
visit. During the home visit, the family support specialist conducted a formal Parent Interest Survey that 
gauged interest in family engagement activities and used a Family Assessment tool which provided 
information about the comprehensive services that a family might need. The Family Assessment tool asked 
the parents to rate how they feel about their housing, exposure to community violence, financial well-being 
of the family and involvement with the criminal justice system. Using this tool, the family support specialist 
helped families develop goals for each month of the school year. Teachers played an important role in 
signaling when a child may need more services by submitting a formal referral. In response to these 
referrals, a comprehensive services team from the school district, which included an educational therapist 
and behavioral support staff person, began working with the child and family. 


In other communities, less systematic methods were used to learn about family needs. In these communities, 
parents were asked about their needs in certain areas during home visits and face-to-face meetings in the 
PEG classrooms. 


5.2.2 Comprehensive Services Provided in PEG Communities in Year 2 


A summary of some of the comprehensive services provided in Year 2 is provided below (Exhibit 6.5), 
followed by profiles of each of the five PEG community’s approach to comprehensive services. 
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Exhibit 6.5: Comprehensive Services Provided by PEG Programs, 2016-17 


Boston (Variation across 
Type ¢ ELPs Lawrence Lowell 


Emotional and mental 
health support 


Physical health 


Material support 


Stress management night led 
by community mental health 
consultant 


Mental health awareness 
event—yoga, mindfulness 
training; led by consultant, 
who also made mental health 
service referrals 


Parent class on healthy 
relationships 

Families referred to dental 
and medical care for the 
children. Vision screening for 


children and follow-up 
referrals for care 


Transportation assistance to 
family engagement events 
using taxi vouchers 


Help for families in 
completing housing 
applications, 


Help connect families to 
outside agencies that offered 
free clothing or ran food 
kitchens or food banks 


Mental health service 
referrals offered for 
children and families 


Parent class on healthy 
relationships 


Dental and physical 
health referrals offered 
to families 


Transportation 
assistance to family 
engagement events 


Mental health service referrals 


Dental, vision and medical 
care services offered directly 
twice a year on site (1 ELP) 


Transportation assistance to 
family engagement events 


Family support specialist 
arranged for clothing and 
household goods from local 
charities that had existing 
partnerships with many of the 
ELP agencies in community. 


Referrals for housing supports 


Housing offered to displaced 
persons (1 ELP) 


Family support specialists 
worked with a family service 
team from the community to 
conduct observations of specific 
children (with the parents’ 
consent) and referred families 
to services 


Mental health team provided 
strategies to teachers for 
supporting children 


Families provided with some 
transportation support in the 
form of a shuttle to take families 
to the program sites. 


Family support specialists 
brought donated clothing to the 
centers for PEG families to pick 
up and when necessary 
delivered food to families who 
struggled to get it on their own. 


Partnership with a local charity 
specializing in trauma-exposure 
to offer mental health services. 
Parents referred to this agency 
for counseling by teachers or the 
family support specialists. 
Agency provided support to 
teachers on supporting children 
who have experienced trauma in 
the classroom 


Employed a health consultant to 
attend to the dental and medical 
health needs of the children. 
Once time during the year 
outside provider of dental, vision, 
hearing, or medical care visited 
Parents provided with clothing 
and shoes/boots obtained from a 
local charity. 


Note. The services listed reflect information gathered via interviews with select program directors and family support staff, and therefore likely are not a comprehensive list of the services offered in 


each community. 
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Springfield 


Springfield offered at least five different kinds of comprehensive services: emotional/mental health support, informational 
support, instrumental support, educational resources, and material support. 


Emotional supports included a stress management night, during which parents received a stress management basket and 
instruction from the community's mental health consultant, and a mental health awareness event, during which parents could 
join a yoga session, make mindfulness charts, and complete other related activities. Child care and dinner were offered during 
both of these events. Parents received referrals from the mental health consultant to outside providers for individualized 
emotional and mental health services. The family support specialist later followed up with individual families to make sure they 
were in fact contacted by a provider. 


The other categories of support (informational, instrumental, educational, and material) were more straightforward. Family 
support specialists managed and regularly updated an information table in each center with pamphlets and other media 
covering topics relevant to the families. Families were also provided with referrals to dental and medical care for the children. 
Children also had access to vision screening and were provided with breakfast, lunch, and a snack during the school day. 


Classes for parents were offered through a Parent Café format during which a parent educator responded to the interests of 
parents to provide classes on healthy relationships and parenting. The PEG coaches also provided a class on using literacy 
activities in the home during a family engagement event. Lastly, this community provided families with transportation 
assistance to family engagement events by giving families vouchers for a taxi, help in completing housing applications, and 
resources to help families connect with outside agencies that offered free clothing and food banks. 


Holyoke 


Holyoke worked closely with Springfield, and its PEG programs therefore participated in or offered many of the same 
comprehensive services as described above. The primary difference between the two communities was the degree to which 
the ELPs worked with the school district. Holyoke had not yet developed as collaborative a relationship with their public school 


district partner as Springfield. This community also highlighted the importance of running mental and medical health referrals 
and services through the families’ health insurance, in order to keep direct costs to the families for accessing these services 
low. 
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Boston 


In Boston, there was less coordination across ELPs, although family support specialists and program directors shared 
information about other community resources for families in monthly PEG leadership meetings. Many programs offered PEG 
families the same comprehensive services that were available to other families attending non-PEG classrooms. Essentially, 
this meant that when centers were run by well-resourced ELPs with established comprehensive service practices, things went 
well. When they were not, outcomes were less certain. 


Most PEG programs provided referrals to families for emotional and mental health support. Most programs also managed an 
information table that kept up-to-date by either a family support specialist on staff or by the program director. In one ELP, 
parents were provided with a family handbook that listed relevant information about community and agency resources in lieu 
of an information table. In another ELP, dental, vision, and medical care services were provided directly to children on site 
twice per year. Another ELP reported screening children and assisting parents in finding medical, dental, or vision care 
homes. 


Family support specialists relied on a range of partners to provide material supports for families. Some PEG programs 
received clothing and household goods from local charities that they had existing partnerships with. Housing assistance was 


generally offered via referrals, with the exception of one ELP which was able to help offer housing for displaced families. 


Lawrence 


In Lawrence, the family support specialists worked with a family service team, which conducted observations of specific 
children (with the parents’ consent) and referred families to services. This team also shared strategies for supporting children 
with teachers, as appropriate, and provided parents (via their children’s backpacks) with educational materials based on the 
needs that parents communicated to family support specialists. Children were referred to an outside agency for physical and 
speech therapy. As was true elsewhere, programs provided children with breakfast, lunch, and a snack during the school day. 


Through a partnership with a local charity, parents were offered a range of classes including computer and ESL instruction. 
Families were provided with some transportation support in the form of a shuttle that was available to take families to the 
program sites. The family support specialists also brought donated clothing to the centers for PEG families to pick up and, 
when necessary, delivered food to families who struggled to access it on their own. 
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Lowell 


Lowell supported parents’ emotional and mental health through a partnership with a local agency specializing in trauma 
exposure. Parents were referred to this agency, as necessary, for counseling by teachers or the family support specialists. 
This organization also provided support to teachers so that children who had experienced trauma could be better supported in 


the classroom. Parents were also offered classes and workshops on financial literacy, and educational materials were 
provided to parents via a resource table. The Lowell PEG site employed a health consultant to attend to the dental and 
medical health needs of the children. Once per year, a provider of dental, vision, hearing, or medical care visited the program. 
These resources were not directly available to parents. Breakfast, lunch and a snack were provided to PEG children. Finally, 
parents were provided with clothing and shoes/boots obtained from a local charity. 


5.2.3 Kindergarten Transition Supports for Families 


In Year 1, PEG programs offered kindergarten registration events that aligned with or were hosted by the 
public school district. In Year 2, these events continued. PEG programs in two communities also started to 
offer more targeted support to families in preparing kindergarten registration forms. Family support 
specialists and other available staff often sat with families and completed the registration forms together. 
One family support specialist mentioned that some of the documents that parents needed for kindergarten 
registration were already in the family’s records, thus reducing the burden placed on the parents to find 
another copy of the child’s birth certificate, for example. In one community, program directors invited 
kindergarten teachers to visit the PEG classrooms and arranged for some children to visit kindergarten 
classrooms in a nearby elementary school. 


5.2.4 Comprehensive Services Challenges 


In one community, providing health services was reported as a challenge because the programs did not have 
nurses on staff. In another community, family support specialists wished that they could offer more classes 
for parents (GED classes, in particular, were singled out as a need). In yet another community, housing 
workshops were identified as a need. While three communities reported that their collaborative relationship 
with the school district was strong and had improved from Year | to Year 2, two communities reported that 
this relationship remained challenging. In both communities, according to program directors, the ELPs and 
the school district did not yet have an effective strategy to coordinate special education screening and 
services. 
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6. Conclusions about Year 2 Implementation Progress and Looking 


Forward to Year 3 


Overall, in Year 2 of implementation, PEG communities and programs appear to have made progress 
towards delivering the key quality components of the PEG model at a consistently high level. 


Educator Supports and Classroom Quality 


In Year 2, community and program leadership made noticeable adjustments to expand professional learning 
supports, such as training and coaching, and offer them more evenly across PEG lead and non-lead teachers. 
Also, efforts were made to adjust the focus of the professional supports to achieve more intentional 
alignment across different types of learning opportunities (i.e. teacher professional development and 
coaching) and instructional approaches (i.e., curriculum and activities). In Year 2, about half of PEG lead 
teachers reported participating in formal teacher networks or professional learning communities, which 
represents a potentially powerful avenue for sustaining improvement in programming and instruction. 


One finding of particular relevance to the full implementation of the teacher support component of the PEG 
model is the mixed perceptions that teachers had regarding the helpfulness of the training and coaching that 
they received. PEG coaches themselves suggested ways that coaching might be improved in future years, 
based on their observations of and interactions with teachers. The recommendations from the PEG coaches 
are important because coaches have a unique perspective on PEG, in that they spend time in classrooms 
working closely with teachers and also retain a global view of the overall program model. 


PEG coaches suggested during interviews that the following steps be taken: 


e Allow more time and space for teachers to leave the classroom for coaching feedback and 
consultation sessions; 

e Provide targeted training to PEG staff, including coaches, about how to work with families 
encountering adversity; 

e Ensure that all coaches are certified in the CLASS and offer training to coaches regarding how to 
improve CLASS scores; 

e Include more opportunities for coaches to meet and collaborate with other coaches in the state; 

e Provide coaches with the opportunity to participate in teacher professional development; and 

e Further increase the amount of coaching available to teachers. 


A key goal of the PEG model is high quality instruction. Observations of the 48 PEG classrooms in Year 
2 showed significant improvements in teachers’ ability to provide emotional support to children and 
create productive classroom learning environments. However, the level of instructional support did not 
change significantly from Year 1 to Year 2. The instructional support finding suggests that there is still 
work to be done to identify which professional learning opportunities will significantly strengthen 
instructional quality (e.g., support for teachers to provide more individualized instruction). This challenge 
is not unique to PEG or Massachusetts, and identifying more effective strategies for building teacher 
capacity in this area represents an opportunity for PEG to contribute to the field more generally. 
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Looking forward, PEG districts and programs may want to: 


e Gather more specific information from teachers on the usefulness of training and coaching and the 
alignment of each to teachers’ individual needs for support and capacity building; 


e Assess if coaches have sufficient time to coach the PEG teachers, given that many coaches have other 
non-PEG coaching responsibilities; and 


e Consider more intensive coaching models focused on the quality of individual teacher-child interactions 
in the classrooms. 


Family Supports 


In Year 2, community and program leadership made similar efforts to try to increase the level of supports 
for families and to introduce more intentionality to the focus of the supports. PEG programs assigned more 
staff to dedicated work with families and offered more enhanced activities and services to help meet their 
needs. There was also more emphasis on family and child mental health and a greater number of services 
provided directly at program sites. 


PEG programs reported focusing less on parent events and focusing more on an overall approach to building 
parent capacities in Year 2. These efforts were not only to increase home support for child learning and 
development, but also to enhance parent mental health as well as parenting skills and broader skills for the 
current job market. A sizeable proportion of teachers (74 percent) reported participation in home visits, 
which raises the potential that deeper connections are being made between PEG classrooms and homes. 


The parent-program connection is an area that merits additional exploration for PEG programs. Although 
challenges remain with engaging and serving all families enrolled in programs, given the other demands 
that many PEG parents have, universal participation may be unrealistic. Yet programs should continue to 
be creative about ways to deliver programs and supports to parents that are flexible and responsive to 
different parent schedules. 


To date, PEG programs report limited capacity and time to systematically track the family supports and 
services provided. Moving forward, EEC may want to consider providing some models for tracking systems 
that community and program leadership can use to more closely monitor the level of supports and services 
that are provided to each family and extent to which services are tailored to family needs. PEG programs 
may want to: 


e Consider conducting regular parent satisfaction surveys, if they are not already doing so, to assess more 
specifically which supports and services are effective and what remaining needs exist; and 


e Assess how to free up more time for family support staff to spend working with families versus focusing 
on other responsibilities. 


A continuing theme of PEG implementation is the notable variation both across and within communities in 
the reach and intensity of supports for educators and for families. More needs to be learned about the extent 
to which variation in supports and services is based on (a) differences across programs in the intentional 
emphasis on different parts of the PEG model; (b) differences within programs in the level of need of 
individual educators and parents; and/or (c) structural and capacity challenges in fully implementing some 
or all of the PEG supports. 


Overall, it appears that all key quality components of the PEG model are being implemented with 
increasing intensity. Whether or not the PEG model impacts child outcomes will be better understood in 
spring 2018, when impact study findings from the PEG evaluation are available. 
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Appendix A: Additional Exhibits 


Exhibit A.1: Hours of Training Received by Non-Lead Teachers (Not Including Individual 
Coaching) by Community, 2016-17 


100% 
80% 
60% 
40% 
20% 
0% 
Overall PEG Boston (n=8) Holyoke (n=7) Lawrence (n=3) Lowell (n=7) Springfield 
(n=38) (n=12) 


n 
- 
7) 
<= 
oO 
o 
7) 
= 
~ 
< 
© 
rey 
_ 
7) 
na 
<x 
-_ 
° 
~ 
c 
vo 
oO 
= 
7) 
a 


Community 


mNone 1-5 hours 6-10 hours 11-20 hours 21-40 hours m™>40 hours 


Exhibit A.2: Coaching Received by PEG Lead Teachers from Public School Coaches by 
Community, 2016-17 
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Exhibit A.3: Coaching Received by PEG Non-Lead Teachers from Public School Coaches by 
Community, 2016-17 
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Exhibit A.4: Focus of PEG Coaching Reported by Lead Teachers by Community, 2016-17 
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Exhibit A.5: Total Hours of Planning Time Received by Lead Teachers in 2016-17 by Community 
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Exhibit A.6: Total Hours of Planning Time Received by Non-Lead Teachers in 2016-17 by 
Community 
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